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REMARKS 

Claims 1-13 are pending in the application. Claims 1-13 are rejected. All 
rejections are respectfully traversed. 

The invention orders multimedia content. Image or video multimedia content 
is segmented to extract objects. Features of the objects are then extracted and 
associated to produce content entities. The content entities are coded to produce 
directed acyclic graphs of the content entities, each directed acyclic graph 
representing a particular interpretation of the multimedia content. Attributes of each 
content entity are measured and the measured attributes are assigned to each 
corresponding content entity in the directed acyclic graphs to rank order the 
multimedia content. 

Claims 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Miller, et al., (News On-Demand for Multimedia Networks - "Miller"), in view of 
Lee, et al., {Querying Multimedia Presentations Based on Content, "Lee"). 

The invention segments multimedia content to extract objects. Miller 
describes a store and forward file transfer system, see Section 4. "The basic 
information unit of the system is the file. A file can contain data like ASCII 
characters, bitmap or structured graphics, a stock market quote or the results of a 
data base search. Files can also contain time-based information such as digitized 
streams of audio or video, or a series of frames stored as animation." see Section 
4.1. 
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In section 5.2, Miller describes organizing multiple files as compound 
stories. Miller never segments the data in the files. Miller is non-analogous art and 
irrelevant to the invention. Miller is devoid of any teaching of segmenting 
multimedia content as claimed. 

Miller generates models that he names media objects. Miller's media object 
is a model including an object control structure (OCS) and an optional media data 
set , i.e., a file. The media objects are combined in binders to create a story. 
Miller's media objects are not the result of segmenting his files. Nowhere in section 
5.2. does Miller segment his files. Applicants respectfully request the Examiner to 
indicate which words in section 5.2 seem to indicate a segmentation. 

His model supports 'composition' and 'sharing', see second paragraph, 
neither is segmenting or extracting features. Objects can be "stored, retrieved, 
replaced, or deleted/' none of these are segmenting or extracting. 

Miller does not extract features fi-om his files to produce content entities., 
measures or assigns attributes, or rank orders anything at all. One of ordinary skill 
in the art would never confiise the model described in Miller with an object 
extracted fi-om multimedia content as claimed. The Applicants request the 
Examiner specifically point out which words in section 5.2 of Miller describe 
segmenting multimedia content in his files or extracting objects. The only 
operations Miller describes in section 5.2 are storing, retrieving, replacing or 
deleting. Those operations are performed on Miller's media object models, see 
section 5.1, not the multimedia content as claimed. 
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Miller never operates directly on multimedia content as in the invention. 
Miller operates on his model. Therefore, Miller can never extract and associate 
features from objects extracted during a segmentation step as in the invention. The 
Applicants request the Examiner specifically point to the words in Miller that mean 
segmented object, extracted feature, or content entity as claimed. 

Miller represents his media object model as a directed acyclic graph (DAG). 
The invention codes features of objects segmented from content, i.e., content entity, 
as DAGs. These are completely unrelated operations. 

Fiulhermore, the Examiner admits that Miller neither measures attributes of 
content entities nor assigns the measured attributes to each corresponding content 
entity in the directed acyclic graphs to rank order the multimedia content as 
claimed. This is tme because there are no content entities in Miller. It would 
appear that Miller does not describe a single element of the invention. 

Lee fails to cure the defects of Miller. Lee starts where the invention ends. 
Lee allows a user to query multimedia presentations based on content. Queries are 
compared to attributes of a representative frame of the multimedia content. 
Claimed is measuring attributes of each content entity, and assigning the measured 
attributes to each corresponding content entity in the directed acyclic graphs to rank 
order the multimedia content. 

Lee never generates content entities from features of extracted objects as 
claimed. Lee simply queries the results. The Applicants request the Examiner 
specifically point to where Lee segments objects, or extracts features and associates 
them to produce content entities as claimed. 
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Lee never measures attributes. Lee compares queries to attributes. The 
Applicants cannot find one word in Lee that teaches measuring attributes as 
claimed. Further, nothing in Lee is rank ordered as claimed. The Applicants have 
thoroughly reviewed Lee, including the section cited by the Examiner, but find no 
teaching whatsoever of the claimed measuring or assigning attributes to content 
entities to rank order the multimedia content as claimed. Lee never rank orders 
anything. The results of Lee's queries are displayed, never rank ordered. 

Figure 2 of Lee does not include the words intensity or order, increasing or 
decreasing, anywhere. Applicants respectfully request the Examiner explain how 
Fig. 2 even suggests intensity attributes or rank ordered attributes. Applicants 
cannot find any words that suggest intensity or rank ordered attributes. 



class PresLGraph type ( 
name SXm$\ 
other aitrihuUi; 
Nodes: {PresJ^ode}; 
Edges: {<PresJv'odo} ] ; 

claas Stream type [ 
name: Siring; 

String; 
prcsJime: Real; 



franics: <I^ianu^>^ 
oojof Jrames: Integer; 
topic: String, 
date: Date; 
oiher Qltrihies]] 

dUm PresJModo : Inhcnts fxom Simm 

type [ 

grapb-m: PrcsjUraph; 
other aUfiht^]; 



daaa FVame type ( 
name: String, 
objects: {C-Object}; 
other attributes ]; 

dasa CObject type { 
name: String; 
type: String; 
framo^jn: mme; 
other atiributei ); 



Notation: **{ Y denotes the set constnictot » and **< >** denotes the sequence cmtstructor. 



Rg. Z Oata moifefog of a praseflmnon ordj^ 



The two special attributes described by Lee are Nodes and Edges. This has 
nothing to do with object direction and there is no suggestion to measure or assign 
the attributes. 
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The same is tme for temporal and Spatial attributes. Lee compares queries to 
attributes. The Examiner is requested to specifically point to where Lee measures 
temporal or spatial attributes, assigns them, or rank orders them. 

The invention traverses the multimedia content and summarizes the 
multimedia content according to the directed acyclic graph and the measured 
attributes assigned to the content entities. As stated above, Lee never measures or 
assigns attributes. Lee uses temporal operators, such as "^'Next, Connected, Untir 
to query a presentation. Lee begins where the invention ends. 

Miller never operates on multimedia content, except for storing, retrieving, 
replacing or deleting. Miller never performs any operation on a three dimensional 
video sequence according to the invention. 

Lee describes DAGs only. Lee only queries DAGs. Lee is silent on content * 
entities according to the invention, segmentation, and measured attributes 
associated with corresponding edges. According to Lee "Each presentation stream 
is a node in the presentation graph and edges describe sequential or concurrent 
playout of streams." See page 363, right col., 3*^/4* paragraphs. The Examiner is 
respectfully requested to explain which words mean measured attributes, 
segmenting, or associating measured attributes with edges. 

Miller never describes content entities produced by extracted and associated 
features as claimed, nor does Miller traverse anything. Therefore, Miller cannot 
teach at least one secondary content entity is associated with a particular content 
entity, and wherein the secondary content entity is selected during the traversing. 
Nor can he teach a summary of the multimedia is a selected permutation of the 
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content entities according to the associated ranks. Miller never ranks or 
summarizes. 

In view of the foregoing, it is respectfully submitted that the application is in 
condition for allowance and an early indication of the same is courteously solicited. 
The Examiner is respectfully requested to contact the undersigned by telephone at 
the below listed telephone nuinber, in order to expedite resolution of any remaining 
issues and further to expedite passage of the application to issue, if any further 
comments, questions ior suggestions arise in connection with the application. 

To the extent necessary, a petition for an extension of time under 37 C.F„R. § 
1 .136 is hereby made. Please charge any shortage in fees due in connection with 
the filing of this paper, including extension of time fees, to Deposit Account 50- 
0749 and please credit any excess fees to such deposit account, 




Respectfully Submitted, 
Mitsubishi Electric 
Research Laboratories, Inc. 



AnOTCW J. Curtin 
Registration No. 48,485 



201 Broadway, 8"^ Floor 
Cambridge, MA 02139 
Telephone: (617) 621-7573 
Facsimile: (617) 621-7550 
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